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There has been a recent emphasis on improving the academic performance of 
students with emotional and behavioral disorders (EBD). Improving the academic 
performance of students with EBD is especially important in the current 
accountability era in which there is much emphasis placed on performance of 
standardized tests. The purpose of this study was to investigate the concurrent and 
predictive validity of curriculum-based measurement (CBM) with students with 
EBD. The Maze, Oral Reading Fluency (ORF), and Written Retell (W-Retell) were 
administered to fifty students’ in grades 6 th through 8 th performance and 
performance on CBM measures examined with the State of Georgia ’s the Criterion 
Referenced Competency Test (CRCT). Results indicated significant correlations 
were found with the Maze and ORF reading measures and the students’ 
performance scores on the CRCT. In addition, of the measures used, the Maze 
explained the most variance in the CRCT scores. Finally, limitations of the study, 
implications for practice, and future research are discussed. 


Improving the academic performance and identifying effective approaches addressing the academic 
progress of students with Emotional and Behavioral Disorders (EBD) is critical (Mooney, Epstein, 
Reid. & Nelson, 2003; Nelson, Benner, Lane, & Smith, 2004). Current academic reforms and 
accountability changes are rapidly occurring with the No Child Left Behind legislation (NCLB, 2002). 
Improved student achievement as measured through statewide achievement tests has become a focal 
point for educational progress. The demand for increased accountability and academic improvement 
has not been singularly exclusive to regular education. Special educators are required to demonstrate 
the effects of their programs and practices (Yssledyke, Thurlow, & Shriner, 1992). Unfortunately, 
statewide achievement tests fail to provide teachers with diagnostic information related to student 
attainment of specific instructional goals (Tindal, & Marston, 1990). The need for alternative 
approaches to monitoring the academic progress has been more widely visible with students who have 
learning disabilities (Fuchs & Fuchs, 1998). However, there has been less discussion regarding 
monitoring of academic behavior for students with EBD. Identifying effective approaches of 
monitoring the progress of students with EBD is needed given a recent national study by Wagner, 
Kutash, Duchnowski, Epstein, and Sumi (2005) that indicated more than 6 in 10 children with EBD 
score in the bottom quartile in reading. Curriculum-based measurement (CBM) is an intervention that 
has been found useful for monitoring the progress of students in special education (Fuchs & Fuchs, 
1991; Shin, Deno, & Espin, 2000) that has direct applications to monitoring the performance of 
students with EBD. 

Fuchs and Fuchs (1991) described CBM as standardized methodology for measuring academic 
performance in the school’s curriculum. Research has indicated that CBM provides accurate 
information about a student’s academic standing and progress, which can be used for a variety of 
educational decisions (Fuchs & Fuchs, 1991). For instance, CBM has been used for (a) screening and 
identifying students for special services (Martson, Mirkin, & Deno, 1984; Shinn, 1989), (b) 
formulating goals and objectives for Individual Education Plans (IEP's) (Deno, Mirkin, & Wesson, 
1983), (c) monitoring student progress and improving educational programs (Deno & Fuchs, 1987), (d) 
transitioning students to less restrictive environments (Fuchs, Fuchs, Hamlett, Phillips, & Bentz, 1994), 
and for (e) evaluating school programs (Germane & Tindal, 1985, Martson, 1988). 


24 



INTERNATIONAL JOURNAL OF SPECIAL EDUCATION 


Vol 22 No2 2007 


With the emphasis on accountability, a growing focus is to use CBM to predict student performance on 
state competency tests of achievement (Tindal, & Marston, 1990). Tindal et. al., (2002) indicated that 
predicting student performance on statewide competency tests of achievement is critical. More efficient 
measures that can provide similar information can be an extremely valuable tool for teachers. Measures 
that give teachers snapshots of students’ conceptual understanding of academic concepts at their grade 
level can fill the need for formative progress monitoring. In addition, justification for predicting 
achievement scores can be found in the school accountability movement that has put a premium on 
educators' providing evidence of student learning (Ysseldyke, Thurlow, & Shriner, 1992). 


CBM was developed to address the major basic skill areas. Examples of using CBM for problem 
solving can be found in reading (Fuchs, & Fuchs, 1991), written expression (Tindal & Marston, 1990), 
spelling (Fuchs, 2004), and math computation (Deno & Fuchs, 1987). CBM was designed to assist 
teachers in monitoring the effectiveness of instruction (Deno, 1985; Deno & Fuchs, 1987). More 
recently, CBM tradition has been expanded to include monitoring students’ acquisition of content in 
secondary content-area classes (Espin & Tindall, 1988), and evaluating student's acquisition of 
phonological skills (Kaminiski & Good, 1998). In the area of reading, for example, the CBM index of 
words read correctly has been shown repeatedly to be a reliable, test-retest reliability ranging between 
.93 and .99, and valid measure, validity coefficients between words read an criterion measures between 
.54 and .91 (Fuchs, Deno, & Mirkin, 1984). 

In our review of the existing literature, no studies that specifically examined the technical adequacy of 
CBM were found. Although not including students with EBD, the following section reviews existing 
literature on (a) Maze, (b) written retell (WR), and (c) oral reading fluency (ORF) to provide a context 
for interpreting the current study focused on students with EBD. The Maze procedure requires the 
selection of a grade-level passage of at least 250-words. The first and last sentences of the passage are 
left intact. Then, a group of words is inserted for every fifth word. The student is asked to select the 
original word by circling it from among three to five distracters within a three to five minute time 
period. The difficulty of Maze varies according to the difficulty of the passage and the difficulty of the 
distracters selected (Howell & Nolet, 2000). Faykus and McCurdy (1998) conducted a study in 
suburban Philadelphia using six students with disabilities. The study used ORF and Maze to assess 
student progress and achievement in reading. Students were examined twice a week for a period of 12 
weeks. Results indicated that oral reading rates might be a more efficient indicator of reading progress 
than Maze within the 12 week time period. Shin, Deno, and Espin (2000) conducted a study in a large 
urban Midwest area using 43-second graders. The study involved ten different Maze passages to assess 
students’ reading performance over a school year. Passages were selected in random from generic 
grade-level reading materials. Results indicated that the Maze task had an alternate-form reliability of 
.81. Additionally, growth rates estimated on repeated Maze measures were positively related to later 
reading performance on a standardized reading test. 

Written Retell (WR) is administered by allowing students to read a grade-level passage for a pre- 
determined time and the student has to retell in writing the story pertaining to the passage that was 
previously read within an allowed time limit (Fuchs, & Fuchs, 1986). Shinn and Good (1992) 
examined WR with 238 students from a mid-sized northwestern city. Students were presented with a 
400-word folktale passage and were given six minutes to read it silently. The students were then given 
a blank sheet of paper and were asked to retell the story on a blank sheet of paper. Results indicated 
significant correlations coefficients with criterion measures for the 3 rd graders at .56 and .50 and 
significant correlations for the 5 ltl graders at .43 and .51. 

Oral reading fluency (ORF) is a short fluency-based measure of oral reading performance based on a 
one-minute timed sample of oral reading behavior. The student is asked to read out loud and the 
examiner marks the number of errors that are made (Martston, 1989). Using standardized procedures, 
students typically read orally from a level of their reading series for a set of repeated, one-minute 
timings. Fewster and Macmillan (2002) conducted a study with 465 middle and high school students 
using CBM oral reading fluency and writing for the screening and placement of students. Students' 
performance scores in the sixth and tenth grade were compared with their year-end English and Social 
Studies grades received in each subject area. The students' grades from their permanent records were 
compared with the CBM measures. CBM probes were developed according to procedures outlined by 
Tilly and Carlson (1992). Results indicated significant correlations between the results on the CBM 
measures and the grade level performance for the students. Words read correctly showed a 
significantly higher correlation with students’ grades than words spelled correctly in written 
expression. There were higher correlations with words read correctly in English than with the Social 
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Studies scores. Also, the correlations were the highest for the students in the eighth grade as compared 
with the other grade levels in middle and high school. Hintze and Shapiro (1997) assessed the effects 
of the curriculum on the technical features of ORF. The study compared the association or criterion- 
related validity of survey-level CBM using literature-based basal reading material and authentic trade 
books. The sample included 57 students enrolled in second, third, and fourth grades from one 
elementary school located in a suburban school district in the Northeast. Results indicated that the 
concurrent validity of CBM oral reading measures were strong with a .665 for the authentic trade book 
series and a .655 for the literature -based basal series. The measure was similar regardless of the reading 
material that was used. In addition, developmental fluency rates were also similar across the two 
curricula. Madelaine and Wheldall (1998) conducted a study using a curriculum-based passage reading 
test for monitoring the performance of low readers. The study examined the criterion validity of the 
Wheldall Assessment of Reading Passages (WARP), 1996 against other established standardized 
reading tests (Neal Analysis of Reading-Revised, 1988; Multilit Word Attack Skills Placement Test, 
Macquarie University Special Education Center, 1996). The study was conducted with 50 students 
from a Sydney independent school. Results indicated moderate to high correlations between the 
WARP, .83 to .87, when comparing phonic word attack skills test, .67 to .72 on reading 
comprehension, and .75 to .78 on the word attack skills. Inter-correlations between the five passages 
were shown to be very high at .94 to .96 demonstrating high alternate form reliability. 

In the field of EBD, there are currently few studies (none were actually located in this review of the 
literature) examining the technical adequacy of CBM with student with EBD. While the reliability and 
validity of many CBM measures has been documented with a range of populations (Shinn, 1989; 
Shinn, 1998), there is a need to further investigate the technical adequacy for CBM for students with 
EBD. Given the importance of improving the academic performance with students with EBD, studies 
are needed to determine whether the technical characteristics of CBM for students with EBD. 
Therefore, the purpose of this study was to examine the convergent and predictive validity of three 
CBM measures of reading (Maze, ORF, and W-Retell) with the state of Georgia’s accountability test, 
the Criterion Referenced Competency Test (CRCT), with a sample of students with EBD at the middle 
school level. 

Method 

Participants and Setting 

The sample consisted of 50 students with emotional and behavior disorders in 6 lh through 8 th grades. A 
middle school from a southern suburban area was chosen to participate in this study. The total middle 
school population is approximately 1,700 students. The ethnicity of the entire school was 
approximately 92% Caucasian, 6% African American, and 2% other. The socioeconomic status of the 
middle school was in the upper middle to high income level with less than 4% of the student 
population below the poverty rate. Fifteen special education teachers served approximately 160 special 
education students of which included 55 students with EBD in a self-contained and resource classroom 
setting. Table 1 provides an overview of their demographic characteristics. 


Table 1 

Sample Student Demographics 


Grade 

Male 

Female 

Caucasian 

African- 

American 

Hispanic 

6 th 

11 

1 

11 

1 

0 

yth 

17 

9 

21 

3 

2 

8 th 

9 

3 

8 

2 

2 

Total 

37 

13 

40 

6 

4 


Administration 

Students were administered the Maze, ORF, and WR. Passages used were developed by Shinn and 
Shinn (2002) and are available to download from the AIMSWEB website (The Maze procedure was 
given to the students within large groups during a time period of 3 minutes per group, while the ORF 
and WR were administered to each student individually with a total test taking time around 3-4 minutes 
per student. The school provided the Criterion Referenced Competency Test data. 

Oral Reading Fluency (ORF). ORF (Shinn 1998) relates to the fluency and accuracy with which a 
student read words. The students were given one minute to read through a 350 word reading passage 
correctly pronouncing words in each sentence. The students' performance was assessed by words read 
correctly minus the number of errors. An error was considered as any mispronunciation of the word or 


26 



INTERNATIONAL JOURNAL OF SPECIAL EDUCATION 


Vol 22 No2 2007 


substitutions, omissions, or 3-second pauses or struggles. ADD 3-4 sentences about technical adequacy 
of Maze from the AIMS WEB site. 

Written Retell (WR). WR was used to assess the accuracy at which a student can recall a reading 
passage that was previous read. To access reading comprehension through recalls, students were given 
one minute to read passages from a 350 word reading passage and then retell in their own words what 
occurred in the passage, without referring back to the passage. The total number of words retold 
correctly was used to assess the student performance. 

Maze. Maze procedure assesses the accuracy and speed at which a student selects a word from a 
multiple word group to properly complete sentences within a passage. The students were given a three- 
minute time limit (Shinn & Shinn, 2002). The first sentence of a passage remained intact, and every 
few words were deleted, and replaced with three choices. Under the time limit, the student selected an 
alternative that meaningfully replaced the blank. A 3-item multiple-choice format with 50 word sets 
was employed with only one choice representing a semantically meaningful replacement. Students' 
indicated their correctly chosen answer by circling one of three choices. 

Criterion-referenced competency test (CRCT). The state of Georgia’s accountability system relies 
heavily on student performance on the CRCT. This criterion referenced test is designed to measures 
student acquisition of the skills and knowledge described in the state standards, known as Quality Core 
Curriculum. The CRCT total scale score for reading was used in this study. 

Data Collection and Analyses 

Descriptive statistics were calculated for each CBM measure and for the CRCT. The Pearson-product 
correlations were calculated between students' total number correct on the CBM measures (Maze, 
ORF, and W-Retell) and the corresponding scores on the CRCT. Also, a forward selection multiple 
regression analysis was used to examine what measure or combination of measures best explained the 
variance in the CRCT scores. 

Results 

Descriptive Statistics 

The means and standard deviations for all three of the CBM measures and Criterion Referenced 
Competency Test are reported in Table 2. The mean score on the curriculum-based measures for the 50 
students with EBD was 21 (SD = 9) for the Maze, 21 (SD = 10) for the Written Retell, and 99 (SD = 
47) for Oral Reading Fluency. Only 26 students in the sample had both CBM and CRCT scores 
available. The mean score for the CRCT was 312 ( SD = 28). 


Table 2 

Descriptive Statistics 


Measure 

N 

M 

SD 

MAZE 

50 

20.55 

8.772 

ORF 

50 

99.45 

46.798 

WRETELL 

50 

21.28 

9.956 

CRCT 

26 

312.15 

27.754 


Correlation and Regression Analysis 

Pearson correlations were calculated to determine the pattern of associations between the measures. A 
correlation was calculated between the CBM measures (Maze, ORF, and Written Retell) and the 
reading section of the CRCT for the middle school grades 6 th , 7 th , and 8 lh . A strong correlation was 
found with the CBM measures (Maze and ORF) and performance scores on the CRCT (r = .439, r 
=.397, p < .05). However, there were no significant differences found between the CRCT and the 
Written Retell. 

Forward selection regression analyses were used to determine the amount of variance in the CRCTs 
could be explained by the CBM measures. Results of the simple regression using the CBM measures 
revealed that the Maze measure was the best predictor. The Maze explained 19.3% of the variance in 
CRCT scores, (see Table 3 next page) 

Discussion 

The concurrent validity between several CBM measures and a standardized assessment used in the 
state of Georgia for high stakes testing was examined. The significance between the CBM measures 
(Maze 
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Table 3 

Correlation Matrix 



1 

2 

3 

4 

l.MAZE 

1 

.391** 

.126 

.439* 

P 

- 

.007 

.397 

.025 

N 

50 

50 

50 

26 

2. ORF 


1 

.599** 

.397* 

P 


- 

.000 

.044 

N 


50 

50 

26 

3. WRETELL 



1 

.148 

P 



- 

.472 

N 



50 

26 

4. CRCT 




1 

P 




- 

N 




26 


Note. *p < .05, two-tailed. **p < 0.01 two-tailed., 


and ORF) and the CRCT aligned with previous research documenting of correlations existing between 
these measures and standardized assessments (Fewster & McMillan, 2002; Shin, Deno, & Espin, 
2000). 

For example, Shin et ai., 2000 found significant correlations between the Maze procedure and student 
performance on a standardized reading assessment. The CBM measures and the CRCT were examined 
and compared to determine their usefulness as indicators of reading progress for students with EBD in 
middle school. The CBM Maze and ORF were the only two measures of significance when being 
compared with other reading subtests on the CRCT. There were no correlations present between the 
CBM-WR and the CRCT being compared in this study. Interesting, the Maze was a better predictor of 
CRCT scores than the ORF measures. 

Overall, the results indicated that the CBM-ORF measure had the highest correlation with the CRCT, 
which may result in a more efficient indicator of decision making at the instructional level for students 
with EBD. According to information from previous studies and investigations (Fuchs et al., 1993; 
Shinn et al., 1992), one reason for this may be that oral reading rate requires fewer component skills 
(e.g., fluent decoding, fluency) than Maze (e.g., decoding, fluency, and comprehension) and as such, 
serves as a better index of reading progress for individuals that have difficulty with reading (Faykus & 
McCurdy, 1998). 

The results of this study suggests that the Maze and ORF reading measures are better predictors of the 
CRCT than Written Retell for students with EBD. However, it is important for educators to consider 
the pros and cons of each measure before deciding on the value of each measure for use in the 
classroom. For example, due to the advantage of being able to listen to the student read using oral 
reading fluency, an educator may be able gain more information in the decision making process to 
provide more specific and appropriate intervention strategies for the classroom. Furthermore, previous 
research indicates (Faykus & McCurdy, 1998) that most teachers preferred the Maze procedure due to 
its efficiency in being able to assess the entire class at one time as compared to the ORF, which can 
only be given to one student at a time. 

Because data from measures of reading rate are often used to inform a broad range of decisions in 
education for monitoring students' progress (Germann & Tindal, 1985), and classification decisions 
(Martson, 1989), this study used this conceptual knowledge as a tool for student performance. We 
reasoned that the students' performance on the CBM measures would be comparable to their 
performance on the standardized measures. The moderate correlation between the oral reading fluency 
and the standardized measure (CRCT) supports this hypothesis which is synonymous with previous 
studies comparing the validity of CBM measures to standardized assessments (Crawford, Tindal, & 
Stieber, 2001; Fuchs, Deno, & Mirkin, 1984; Tindal & Marston, 1990). 
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Limitations 

The results of this study provide some support for CBM to assess the reading progress of students with 
EBD. However, there were small numbers of participants prohibiting a larger comparison and more 
statements as to the validity of the CBM measures when being compared with the standardized 
measures. Another limitation of this study was that one teacher administered all of the measures. 
Although the teacher was trained in the administration of reading timings and proficient in the 
administration of the statewide assessments, no formal reliability checks were performed to assure 
standardization. However, because all of the students received their education in the same school 
setting, the same teacher collected the data for all of the subjects. Another shortcoming of this study 
was the relatively small sample size. There were only 50 students included in this study. The small 
sample size increased the standard error of measurement and prohibits our ability to make 
generalizations to a larger population. 

Implications for Practice 

In this study, we expanded on the conditions of use for CBM being compared with standardized and 
statewide measurements further supporting CBM use as a valid measurement tool for comparisons with 
previous student performance and potentially predicting future performance students with EBD 
(Crawford, Tindall, & Stieber, 2001). The data presented in this study demonstrates that CBM’s are 
sensitive enough to be used in comparison with standardized and state assessments in assessing a 
student's reading ability. Teachers must recognize CBM as a general indicator and interpret its results 
in combination with various other types of assessments and data (Fewester & McMillan, 2002). CBM 
results can be helpful in identifying problems that warrant future investigations for the students in the 
classroom. The information from CBM measures can be extended into the development of appropriate 
intervention plans directly related to student's with EBD in the classroom. These applications make 
CBM measures an attractive component for classroom teachers and school personnel in monitoring 
student performance for students’ with emotional and behavior disorders. 

Results of the current study are consistent with those of previous studies that have suggested that 
assessing reading comprehension may be similar to assessing oral reading fluency (Deno, 1985). The 
students that exhibit poor oral reading skills have comprehension skills that are often equal to or lower 
than their reading fluency levels (Hinzte & Shapiro, 1997). The students that may be referred for 
reading problems and found to have oral reading fluency difficulties implies that a separate assessment 
of comprehension may not be warranted. A screening for reading comprehension may be incorporated 
by selecting a simple Maze passage and procedure for students who may exhibit adequate decoding 
and fluency skills. Another practical use for CBM would be to formulating goals and objectives for 
Individual Education Plans (Deno, Mirkin, & Wesson, 1984). Curriculum-based measures can provide 
immediate feedback for a teacher to see if an intervention is having a positive effect and if not, to make 
a modification in a plan that was previously established. The measures can be instructional in pointing 
out areas of weakness to be addressed specifically for each student through the goals and objectives of 
the IEP. Furthermore, CBM procedures involve the direct observation of student behavior and use 
single case analytic procedures that are similar to characteristics of applied behavior analysis (Deno, 
1985). 

Future Research 

There are several implications for future research with the technical adequacy of CBM for students 
with EBD. More studies are clearly needed examining the technical adequacy of CBM for students 
with EBD. Students with EBD are variable populations with many unique characteristics that may 
affect the technical adequacy of the measures. Having additional studies will assist in buttressing the 
proposition that CBM can be used to accurately inform progress for students with EBD, especially 
those who have academic deficits. Moreover, some students with EBD may be displaying their 
problem behaviors to escape and avoid difficult academic tasks. Once the technical adequacy of CBM 
measures have been established for instructional problem-solving, research will be needed to examine 
how these measures may be used within a function-based approach to intervention planning (Espin & 
Tindal, 1998). 

Conclusions 

The use of CBM measures as a source of information in screening and eligibility decisions meets 
current functional assessment requirements. The information from the CBM measures can provide 
some objective data that can be incorporated with other assessments in developing the appropriate 
intervention plans directly relating to the current academic curriculum for students' with emotional and 
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behavior disorders. The assessment process can be an important aspect to monitor the progress of 
student performance and assessing the effectiveness of a particular education program. We believe that 
these components are what make CBM measures such an attractive component for classroom teachers 
and school personnel in educating students with EBD. 
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